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FOREWORD 

The  Operations  Research  and  Economic  Analysis  Office,  Directorate 
of  Resources  Namagement,  Defense  Fuel  Supply  Center,  was  tasked 
with  evaluating  the  multiple  forecasting  model  developed  in 
Multiple  Forecasting  Techniques.  Defense  Logistics  Agency  (DLA) 
Operations  Research  and  Economic  Analysis  Office  Study  DLA-91- 
P90053,  as  a  predictor  of  fuel  requirements. 

The  authors  of  this  paper  are  indebted  to  the  authors  of  Multiple 
Forecasting  Techniques  for  their  pioneering  work  in  the  use  of 
these  techniques  and  for  sharing  computer  programs  and  other 
insights  into  the  problems  of  multiple  forecasting. 

We  are  also  indebted  to  the  Inventory  Management  Division, 
Directorate  of  Supply  Operations,  Defense  Fuel  Supply  Center,  for 
their  assistance  in  formulating  the  initial  concepts  of  this 
study  and  for  their  assistance  in  acquisition  of  data. 


EXECUTIVE  SUMMARY 


Defense  Management  Review  Decision  (DMRD)  901  (Reduce  Supply 
System  Costs)  led  the  Defense  Logistics  Agency  (DLA)  to  institute 
a  study  under  the  auspices  of  the  Requirements  Forecast  Working 
Group  (RFWG)  on  the  use  of  multiple  forecasting  techniques  to 
forecast  demand  for  items.  The  initial  work,  done  at  DLA 
Operations  Research  and  Economic  Analysis  Management  Support 
Office  (DLA-DORO) ,  was  applied  to  construction,  electronics, 
general,  industrial,  and  medical  commodities.  These  commodities 
are  all  managed  within  the  Standard  Automated  Materiel  Management 
System  (SAMMS) .  This  initial  study  showed  the  potential  for 
inventory  safety-level  reductions  of  $42  million.  RFWG  requested 
an  extension  of  this  methodology  to  fuels,  which  is  managed  using 
the  Defense  Fuel  Automated  Management  System  (DFAMS) . 

This  study  examines  the  operation  of  the  multiple  forecasting 
model  on  fuel  sales  data  and  demonstrates  improvements  in 
forecast  accuracy  gained  through  the  use  of  the  multiple 
forecasting  technique.  More  accurate  forecasts  of  demands  at 
Defense  Fuel  Supply  Points  and  other  customer  locations  will 
allow  DFSC  to  better  support  them  by  lower-cost  fuel  movements, 
fewer  emergency  shipments  and  fuel  purchases,  and  perhaps  lowered 
inventory  levels  resulting  from  reduced  safety  level 
requirements.  This  report  recommends  that  the  Fuels  Multiple 
Forecasting  Model  developed  here  be  used  on  a  test  basis  by  DFSC. 
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SECTION  1 
INTRODUCTION 

1.1  BACKGROUND 


The  Defense  Management  Review  Decision  (DMRD)  901  (Reduce  Supply 
System  Costs)  led  the  Defense  Logistics  Agency  (DLA)  to  institute 
a  study  under  the  auspices  of  the  Requirements  Forecast  Working 
Group  (RFWG)  on  the  use  of  multiple  forecasting  techniques  to 
forecast  demand  for  items.  The  initial  work,  done  at  DLA 
Operations  Research  and  Economic  Analysis  Management  Support 
Office  (DLA-DORO) ,  was  applied  to  construction,  electronics, 
general,  industrial,  and  medical  commodities.  These  commodities 
are  all  managed  within  the  Standard  Automated  Materiel  Management 
System  (SAMMS) .  This  initial  study  showed  the  potential  for 
inventory  safety-level  reductions  of  $42  million.  RFWG  requested 
an  extension  of  this  methodology  to  fuels,  which  is  managed  using 
the  Defense  Fuel  Automated  Management  System  (DFAMS) . 

1.2  PURPOSE 


The  purpose  of  this  study  is  to  evaluate  the  operation  of  the 
multiple  forecasting  model  on  DFSC  fuel  sales  data  and  to 
evaluate  the  criteria  used  to  select  the  optimal  forecasting 
techniques  within  the  model. 

1.3  OBJECTIVES 


The  objectives  of  this  study  are: 

(1)  Evaluate  the  forecasting  techniques  used  and  the  model 
results  for  useability  by  inventory  managers. 

(2)  Provide  recommendations  for  implementation  of  the 
Multiple  Forecasting  Model  at  DFSC. 
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1.4 


SCOPE  OF  STUDY 

(1)  Data  was  gathered  from  OFAHS. 

(2)  Only  sales  of  fuels  procured  under  bulk  fuel  purchasing 
contracts  were  forecasted. 

(3)  Only  those  forecasting  methods  included  in  the  original 
Multiple  Forecasting  Techniques  study  (DLA-91-P90053)  were 
considered  in  this  study  (see  Section  2.1). 

(4)  Data  was  consolidated  at  the  Defense  Fuel  Supply  Point 
(DFSP)  or  Department  of  Defense  Activity  Address  Code  (DODAAC) 
level . 
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SECTION  2 
METHODOLOGY 

2.1  DESCRIPTION  OP  THE  MULTIPLE  FORECASTING 

MODEL 

DLA's  Multiple  Forecasting  Model  is  based  on  the  premise  that  one 
forecasting  technique  will  prove  to  be  best  when  tested  over  a 
given  set  of  actual  data  points  and  the  same  technique  will  be 
best  when  used  to  generate  the  forecast  for  future  points.  The 
model  uses  20  quarters  of  data;  12  quarters  for  initialization; 
the  next  4  quarters  to  test  the  various  forecasting  techniques; 
and  then  the  last  4  quarters  to  evaluate  the  results  of  the 
selected  forecasting  technique  against  a  forecast  of  demand 
currently  available  in  SAMMS. 

The  forecasting  techniques  tested  by  the  model  include: 

(1)  Four  Quarter  Moving  Average 

(2)  Eight  Quarter  Moving  Average 

(3)  Lagged  moving  average  based  on  autocorielations 

(4)  Single  exponential  smoothing  with  alpha  =  O.l 

(5)  Single  exponential  smoothing  with  alpha  =  0.2 

(6)  Double  exponential  smoothing  with  alpha  =  0.1 

(7)  Double  exponential  smoothing  with  alpha  =  0.2 

(8)  Adaptive  response  single  exponential  smoothing 

(9)  Adaptive  response  exponential  smoothing  with  a 
smoothed  alpha 

(10)  Holt's  double  exponential  smoothing 

(11)  Winter's  triple  exponential  smoothing  with  alpha  =0.1 

(12)  Winter's  triple  exponential  smoothing  with  alpha  =  0.2 

(13)  Simple  linear  regression 

(14)  Last  demand 

(15)  Year  ago  demand 

(16)  SAMMS  Double  exponential  smoothing  with  alpha  =  0.2 

(17)  Conditional  probability  model  based  on  clustering 

(18)  Nonlinear  regression 
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DLA's  Multiple  Forecasting  Model  used  three  different  measures  of 
error  in  selecting  the  appropriate  forecasting  technique: 

•  Mean  absolute  deviation  (MAD) ,  the  average  of 
forecasting  errors 

•  Mean  squared  error  (MSE) ,  the  average  of  squared 
forecasting  errors 

•  Total  absolute  error  (TAE) ,  the  difference  between  the 
svim  of  the  forecasts  and  the  sum  of  demands 


The  model  then  used  the  modified  index  of  predictive  efficiency 
(MIPE) ,  a  relative  measure  that  compares  two  forecasts,  to 
evaluate  the  selected  forecasting  techniques  against  the  forecast 
in  SAMMS. 

The  DLA  model  used  Theill's  bias  proportion  to  eliminate  any 
forecasting  technique  that  exceeds  a  threshold  limit. 

2.2  FUELS  MULTIPLE  FORECASTING  MODEL 


Using  DLA's  Multiple  Forecasting  Model  as  a  guide,  a  similar 
model,  the  Fuels  Multiple  Forecasting  Model,  was  created  using  a 
LOTUS  spreadsheet.  The  Fuels  Multiple  Forecasting  Model  uses  20 
quarters  of  data  in  the  same  way  as  the  Multiple  Forecasting 
Model.  Twelve  quarters  are  used  to  initialize  the  model.  Four 
quarters  of  data  are  used  to  test  the  fifteen  forecasting 
techniques  to  determine  which  technique  to  select  as  the  "best”, 
for  the  particular  data  set.  Finally,  the  last  4  quarters  of 
data  are  used  to  test  the  advantages  of  using  the  chosen 
forecasting  method  against  an  existing  forecast. 
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2.3 


FORECASTING  TECHNIQUES 


The  forecasting  techniques  tested  by  this  model  include: 


(1)  Four  Quarter  Moving  Average 

(2)  Eight  Quarter  Moving  Average 

(3)  Lagged  moving  average  based  on  autocorrelations 

(4)  Single  exponential  smoothing  with  alpha  =0.1 

(5)  Single  exponential  smoothing  with  alpha  =  0.2 

(6)  Double  exponential  smoothing  with  alpha  =  0.1 

(7)  Double  exponential  smoothing  with  alpha  =  0.2 

(8)  Adaptive  response  single  exponential  smoothing 

(9)  Adaptive  response  exponential  smoothing  with  a  smoothed 
alpha 

(10)  Holt's  double  exponential  smoothing 

(■^1)  Winter's  triple  exponential  smoothing  with  alpha  =  0.1 

(12)  Winter's  triple  exponential  smoothing  with  alpha  =  0.2 

(13)  Simple  linear  regression 

(14)  Last  demand 

(15)  Year  ago  demand 


2.4 


FORECASTING  TECHNIQUE  COMPARISONS 


It  must  be  noted  that  there  are  three  fewer  forecasting 
techniques  used  in  the  Fuels  Multiple  Forecasting  Model  than  in 
the  Multiple  Forecasting  Model.  The  SAMMS  model  was  deemed  to  be 
extraneous  to  the  DFAMS  commodity.  The  conditional  probability 
model  was  excluded  because  the  data  from  DFAMS  did  not  have  the 
large  number  of  null  values  characteristic  of  the  SAMMS  data. 

The  nonlinear  regression  method  is  merely  regression  using  a 
logarithmic  transformation  of  the  data. 


2.5  SELECTION  OF  BEST  TECHNIQUE 

The  choice  of  the  "best”  forecasting  technique  for  a  data  set  was 
based  on  the  same  two  criteria  as  the  Multiple  Forecasting  Model: 
bias  and  accuracy.  The  bias  measure  was  set  at  the  0.2  threshold 
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as  in  the  Multiple  Forecasting  Model.  Any  forecasting  technigue 
whose  bias  proportion  exceeded  this  threshold  was  rejected.  The 
accuracy  measures  used  were  the  MAD,  MSB  and  the  TAB.  The  "best” 
forecasting  technigue  was  the  one  that  was  both  unbiased  and 
highly  accurate.  In  the  event  accuracy  measures  prescribe 
different  forecasting  technigues,  the  model  defaults  to  the 
technigue  with  the  lowest  MAO. 

In  the  determination  of  the  relative  advantages  of  the  Fuel 
Multiple  Forecasting  Model  over  the  existing  forecast  in  DFAMS, 
the  MIPB  plays  a  key  role.  This  index  compares  the  two  forecasts 
via  a  ritio  of  TABs  for  the  forecasts. 
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SECTION  3 
DATA 


3.1  QUARTERLY  SDMMARY  OF  SALES 

The  Fuels  Multiple  Forecasting  Model  was  evaluated  against 
quarterly  summary  of  sales  data  by  receiving  activity  taken  from 
DFAMS.  The  DFAMS  report  shows  four  quarters  of  demand,  the 
previous  yearns  demand,  ard  a  forecast  of  the  next  year's  demand. 
The  reports  for  the  fiscal  years  1985  through  1990  were  used. 
Fiscal  year  1991  was  excluded  because  of  the  possibility  of 
distortion  due  to  Desert  Shield/Desert  Storm  demand. 


3.2 


SELECTION  OF  TEST  SAMPLES 


The  model  was  tested  on  a  sample  of  data  from  the  targeted  years. 
A  randomized  choice  of  locations  for  each  of  the  major  product 
types  was  made.  Major  product  types  were:  Navy  Distillate  Fuel 
Oil  (DFW) ,  Regular  Grade  Diesel  Fuel  Oil  (DF2) ,  Navy  Distillate 
Fuel  Oil  (F76) ,  and  Aviation  Turbine  Fuels  (JP4,  JP5,  and  JP8) . 
Together  these  product  types  account  for  over  90  percent  of  DFSC 
business. 

A  total  of  29  locations  were  selected  for  testing.  The  breakout 
by  product  type  was: 

Product  Number  of  Locations 


DFW  2 

DF2  3 

F76  6 

JP4  9 

JP5  8 

JP8  1 


In  one  case,  a  location  chosen  as  a  JP4  test  bed  converted  to  JP8 
usage  during  the  5  year  period.  In  this  case,  the  data  strings 
for  the  commodities  were  combined  and  the  model  operated  on  the 
complete  string. 
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SECTION  4 
ANALYSIS 

4.1  PERFORMANCE  OF  THE  PSELS  MULTIPLE  FORECASTING 

MODEL 

4.1.1  FORECASTING  TECHNIQUES  SELECTED 

The  frequency  of  forecasting  technique  selection  for  the  test 
data  by  the  Fuels  Multiple  Forecasting  Model  was  as  follows: 

Forecasting  Technique  Selected  Frequency 


Holtzs  double  exponential  smoothing  11 
Last  demand  4 
Year  ago  demand  3 
Lagged  moving  average  based  on  autocorrelation  2 
Double  exponential  smoothing,  alpha  =0.1  2 
Adaptive  response  single  exponential  smoothing  2 
Winter's  triple  exponential  smoothing,  alpha  =0.1  1 
Double  exponential  smoothing,  alpha  =0.2  1 
No  Method  (see  Section  4.1.2)  3 


4.1.2  BIAS 

In  three  cases  the  Fuels  Multiple  Forecasting  Model  rejected  all 
of  the  forecasting  methods  because  the  bias  proportions  exceeded 
the  0.2  threshold.  In  all  three  cases  these  were  locations  where 
fuel  use  changed  suddenly  and  radically  from  one  year  to  the 
next . 


4.2 


COMPARISON  OF  THE  MODEL  WITH  THE  CURRENT 
FORECAST 


Of  29  locations  tested,  the  Fuels  Multiple  Forecasting  Model 
outperforms  the  forecast  on  the  DFAMS  report  in  21  cases.  In  2 
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cases  the  report  forecast  was  closer  to  the  actual  data,  and  in  3 
cases  the  accuracy  measures  were  inconclusive  (split  between  the 
two  forecasts) .  Overall  percentage  improvements  in  accuracy 
measures  for  the  Fuels  Multiple  Forecasting  Model  over  the  DFAMS 
report  forecast  are:  MAD,  24.1  percent;  MSB,  22.0  percent;  and 
TAB,  45.6  percent. 

Results  are  similar  using  the  MIPB;  the  Fuels  Multiple 
Forecasting  Model  outperforms  the  forecast  on  the  DFAMS  report  in 
21  cases,  while  the  DFAMS  report  is  better  in  5  cases. 

4.3  COMPARISON  OF  THE  MODEL  WITH  THE  COMMERCIAL 

PACKAGE 


To  establish  some  model  validity  with  respect  to  a  commercially 
available  forecasting  package,  the  data  was  run  through  the 
forecasting  package  called  AutoCast.  This  comparison  was  done 
using  a  non-seasonal,  constant-level,  exponentially  smoothed 
model  from  AutoCast.  The  package  is  capable  of  other  options, 
but  for  this  investigation,  this  model  was  chosen  as  the  most 
likely,  generic  model.  In  this  comparison,  the  Fuel  Multiple 
Forecasting  Model  proved  more  accurate  15  times,  AutoCast  was 
more  accurate  7  times,  and  the  results  were  mixed  4  times.  The 
results  were  similar  using  the  MIPB.  The  index  for  the  Fuels 
Multiple  Forecasting  Model  was  best  19  times  versus  7  times  for 
AutoCast. 
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SECTION  5 
RESULTS 


5.1 


CONCLUSIONS 


The  multiple  forecasting  technique  often  provides  a  more  accurate 
forecast  than  either  the  current  system  or  the  AutoCast  model. 

5.2  BENEFITS 


More  accurate  forecasts  of  demands  at  DFSPs  and  other  customer 
locations  will  allow  DFSC  to  better  support  them  by  lower-cost 
fuel  movements,  fewer  emergency  shipments  and  emergency  fuel 
purchases,  and  perhaps  lowered  inventory  levels  resulting  from 
reduced  safety  level  requirements.  Certainly  the  ability  to 
accurately  forecast  demands  would  give  DFSC  the  ability  to 
challenge  annual  requirements  submissions;  especially  when  those 
requirements  differ  significantly  from  the  Fuels  Multiple 
Forecasting  Model. 


5.3 


RECOMMENDATIONS  AND  SUGGESTIONS  FOR  FURTHER 
RESEARCH 


Specific  recommendations  stemming  from  this  project  are  the 
following: 

•  The  customer.  Inventory  Management  Division,  Directorate 
of  Supply  Operations,  DFSC-OI,  use  the  Fuels  Multiple 
Forecasting  Model  on  a  test  basis. 

•  An  effort  should  be  made  to  determine  if  the  alpha 
values  used  in  the  exponential  smoothing  techniques  were 
the  best  possible. 

•  Other  forecasting  techniques  should  be  considered  for 
inclusion  into  the  model.  The  Box-Jenkins  technique  was 
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used  at  DFSC  with  some  success  several  years  ago  and 
should  be  reconsidered. 

•  As  in  the  DLA  Multiple  Forecasting  Techniques  study ^  an 
investigation  be  made  into  the  effects  of  implementing 
this  model  as  a  dynamic  forecasting  model,  allowing  the 
forecasting  technique  to  change  for  each  quarterly 
forecast. 
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APPENDIX  A 
EQUATIONS 


A-1 


APPENDIX  A 
EQUATIONS 

The  following  is  a  mathematical  representation  of  the  forecast 
accuracy  measures,  bias  measures,  and  forecasting  techniques 
referred  to  in  this  report. 

A-1.1  PORECA8T  ACCURACY  MEASURES 

A-1.1.1  MEAN  ABSOLUTE  DEVIATION  (MAD) 

n 

Y,  ABS(A,-F,) 

mad  =  - 

n 


where:  F,  =  forecast  for  time  t 

At  =  actual  demand  for  time  t 
n  =  number  of  time  periods 

A-l.1.2  MEAN  SQUARED  ERROR  (MSE) 

E 

mse  =  - 

n 

where:  F,  =  forecast  for  time  t 

A,  =  actual  demand  for  time  t 
n  =  number  of  time  periods 
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A-1.1.3 


TOTAL  ABSOLUTE  ERROR  (TAE) 


where : 


A-1.1.4 


where; 


A-1.2 


where ; 


N 

TAE  =  ABSj^  (At-Ft) 
t=l 


F,  =  forecast  for  time  t 
At  =  actual  demand  for  time  t 
n  =  number  of  time  periods 

MODIFIED  INDEX  OF  PREDICTIVE  EFFICIENCY 
(MIPE) 

E  ~  E 

MIPE  =  100  ♦  [ - - - - — j 

~  (Es  +  EJ 

E,  =  TAE  of  basic  forecast 
E„  =  TAE  of  model  forecast 


BIAS  MEASURE 


U  = 


RMSE 


2 


F  =  average  forecast 
A  =  average  actual  demand 

RMSE  =  square  root  of  the  MSE 
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A>1.3.S  LAGGED  TWO  QUARTER  MOVING  AVERAGE 

Lagged  two  quarter  moving  average  determines  if  there  is  any 
relationship  between  quarters  two,  three,  or  four  intervals 
apart.  Autocorrelations  are  computed  for  these  lags  and  a 
ninety-five  percent  confidence  interval  is  used  to  establish 
significance.  Decision  rules  are: 
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If  the  second  autocorrelation  is  positively  significant 
then: 


t*l 


(Xt-x  *  ^t-3) 
2 


•  If  the  third  autocorrelation  is  positively  significant 
then: 


c+i 


*  X,.^) 

2 


If  the  fourth  autocorrelation  is  positively  significant 
then: 


F 


t+i 


{X^.3  ^t-7) 

2 


•  If  none  of  the  above  are  true  then: 


C+l 


{X,  *  X,.^) 
2 


A-1.3.6  SINGLE  EXPONENTIAL  SMOOTHING 

=  ax^+il  -  a)F, 
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A-1.3.7 


DOUBLE  EXPONENTZlkL  SMOOTHING 


where : 


A-1.3.8 


where ; 


=  aJft  +  (1  - 
=  aS'^  +  (1  - 
=  2S'^  -  S"^ 

■  'Ti4io> 

ft..  ‘ 

S'  =  single  smoothed  quantity 
S'*  =  double  smoothed  quantity 
ot  =  smoothing  parameter 
m  =  length  of  forecasting  horizon 

HOLT'S  DOUBLE  EXPONENTIAL  SMOOTHING 

5c  =  aXt  +  (1  -  a)  (5c.i  +  2?c.i) 
b,  =  5(5c  -  5c.i)  +  (1  - 
Pt*m  =  s,*  b^ 

S  =  smoothed  quantity 
b  =  trend  component 
a  =  smoothing  parameter 
S  =  smoothing  parameter  for  trend 
m  =  length  of  forecast  horizon 
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A-1.3.9 


WIMTER'S  TRIPLE  EXPONENTIAL  SMOOTHING 


A-1.3.10 


t>c 


=  ^  +  (1  -  a) 

=  fits,  -  5,.,)  +  (1  -  6)b,.^ 

Jt  =  +  <1  -  P)rt-i 


where :  S 

b 
I 
a 
S 

0 

m 


smoothed  quantity 

trend  component 

seasonal  component 

smoothing  parameter 

smoothing  parameter  for  trend 

smoothing  parameter  for  seasonality 

length  of  forecasting  horizon 


TRI6G-LEACH  ADAPTIVE  RESPONSE  RATE  EXPONENTIIO. 
SMOOTHING 

e,  =  j:,  -  f/ 

E,  =  Pe,  (1  -  P)Ft.i 
M,  =  P|e,|  +  (1  -  P)//,.i 

“t  =  I— I 

Lagged  a  Trigg-Leach  Model: 

■^t+l  ~ 

Smoothed  a  Trigg-Leach  Model: 

a't,  =  6  («£,)  +  (1  -  5)  a't-i 

^t*i  =  «'c^c  +  (1  - 

where:  e  =  forecast  error 

E  =  smoothed  forecast  error 
0  =  smoothing  parameter  for  error 
M  =  smoothed  absolute  error 
a  =  computed  smoothing  parameter 
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A-1.3.11 


SIMPLE  LINEAR  REGRESSION 


where : 


n  E-Sr"  -  (£■« 


a  = 


=  a  +  hm 


Y  =  the  period  of  the  time  series 
n  =  the  number  of  quarters  included 
m  =  length  of  forecasting  horizon 
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APPENDIX  B 
LlS^f  Oi?  ACRONYMS 
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APPENDIX  B 
LIST  OF  ACRONYMS 


DFAMS 

DFSC 

DFSP 

DFW 

DF2 

DLA 

DMRD 

DODAAC 

DORO 

F76 

JP4 

JP5 

JP8 

MAD 

MIPE 

MSE 

RFWG 

SAMMS 

TAE 


Defense  Fuel  Automated  Management  System 

Defense  Fuel  Supply  Center 

Defense  Fuel  Supply  Point 

Navy  Distillate  Fuel  Oil 

Regular  Grade  Diesel  Fuel  Oil 

Defense  Logistics  Agency 

Defense  Management  Review  Decision 

Department  of  Defense  Activity  Address  Code 

DLA  Operations  Research  and  Economic  Analysis 

Management  Support  Office 

Navy  Distillate  Fuel  Oil 

Aviation  Turbine  Fuel,  Grade  JP4 

Aviation  Turbine  Fuel,  Grade  JP5 

Aviation  Turbine  Fuel,  Grade  JP8 

Mean  Absolute  Deviation 

Modified  Index  of  Predictive  Efficiency 

Mean  Squared  Error 

Requirements  Forecast  Working  Group 
Standard  Automated  Materiel  Management  System 
Total  Absolute  Error 
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